72 research outputs found
Multilabel Consensus Classification
In the era of big data, a large amount of noisy and incomplete data can be
collected from multiple sources for prediction tasks. Combining multiple models
or data sources helps to counteract the effects of low data quality and the
bias of any single model or data source, and thus can improve the robustness
and the performance of predictive models. Out of privacy, storage and bandwidth
considerations, in certain circumstances one has to combine the predictions
from multiple models or data sources to obtain the final predictions without
accessing the raw data. Consensus-based prediction combination algorithms are
effective for such situations. However, current research on prediction
combination focuses on the single label setting, where an instance can have one
and only one label. Nonetheless, data nowadays are usually multilabeled, such
that more than one label have to be predicted at the same time. Direct
applications of existing prediction combination methods to multilabel settings
can lead to degenerated performance. In this paper, we address the challenges
of combining predictions from multiple multilabel classifiers and propose two
novel algorithms, MLCM-r (MultiLabel Consensus Maximization for ranking) and
MLCM-a (MLCM for microAUC). These algorithms can capture label correlations
that are common in multilabel classifications, and optimize corresponding
performance metrics. Experimental results on popular multilabel classification
tasks verify the theoretical analysis and effectiveness of the proposed
methods
SEVEN: Deep Semi-supervised Verification Networks
Verification determines whether two samples belong to the same class or not,
and has important applications such as face and fingerprint verification, where
thousands or millions of categories are present but each category has scarce
labeled examples, presenting two major challenges for existing deep learning
models. We propose a deep semi-supervised model named SEmi-supervised
VErification Network (SEVEN) to address these challenges. The model consists of
two complementary components. The generative component addresses the lack of
supervision within each category by learning general salient structures from a
large amount of data across categories. The discriminative component exploits
the learned general features to mitigate the lack of supervision within
categories, and also directs the generative component to find more informative
structures of the whole data manifold. The two components are tied together in
SEVEN to allow an end-to-end training of the two components. Extensive
experiments on four verification tasks demonstrate that SEVEN significantly
outperforms other state-of-the-art deep semi-supervised techniques when labeled
data are in short supply. Furthermore, SEVEN is competitive with fully
supervised baselines trained with a larger amount of labeled data. It indicates
the importance of the generative component in SEVEN.Comment: 7 pages, 2 figures, accepted to the 2017 International Joint
Conference on Artificial Intelligence (IJCAI-17
DetectGPT-SC: Improving Detection of Text Generated by Large Language Models through Self-Consistency with Masked Predictions
General large language models (LLMs) such as ChatGPT have shown remarkable
success, but it has also raised concerns among people about the misuse of
AI-generated texts. Therefore, an important question is how to detect whether
the texts are generated by ChatGPT or by humans. Existing detectors are built
on the assumption that there is a distribution gap between human-generated and
AI-generated texts. These gaps are typically identified using statistical
information or classifiers. In contrast to prior research methods, we find that
large language models such as ChatGPT exhibit strong self-consistency in text
generation and continuation. Self-consistency capitalizes on the intuition that
AI-generated texts can still be reasoned with by large language models using
the same logical reasoning when portions of the texts are masked, which differs
from human-generated texts. Using this observation, we subsequently proposed a
new method for AI-generated texts detection based on self-consistency with
masked predictions to determine whether a text is generated by LLMs. This
method, which we call DetectGPT-SC. We conducted a series of experiments to
evaluate the performance of DetectGPT-SC. In these experiments, we employed
various mask scheme, zero-shot, and simple prompt for completing masked texts
and self-consistency predictions. The results indicate that DetectGPT-SC
outperforms the current state-of-the-art across different tasks.Comment: 7 pages, 3 figure
- …